Apparatus for remote control of a microscope

ABSTRACT

An apparatus and method acquires and stores multiple resolution images from a specimen on a support and provides to the user a low magnification, reconstructed macro image of the entire specimen, or a large portion thereof, to aid the person in selecting points of interest to be viewed or analyzed at higher magnifications and resolution. The reconstructed image is formed of a large number of tiled, stored images which are coordinated and assembled to form the macro image of the specimen which is displayed on a monitor. Preferably, the stored, reconstructed image is reduced further in size by a software system before it is displayed to the user. The display may be on a local monitor over a local area network or sent over the Internet to the user who is typically a pathologist. The user selects by a marker such as a cursor the defined area of interest or region and then views higher magnification images or has them analyzed. Preferably, the pathologist can scroll to shift digitized, adjacent image tiles into view on the monitor. A fully computer-controlled microscope is used to acquire and store the digitized images and the illustrated microscope can be remotely controlled to change objective lenses, focus, light intensity, filters, field diaphragm, and to shift the microscope stage by a controller.

[0001] This is a continuation of a co-pending application, U.S. patentapplication Ser. No. 10/008,519, filed Nov. 13, 2001, which is acontinuation of U.S. patent application Ser. No. 09/752,022 filed Dec.28, 2000, which is a divisional application of U.S. patent applicationSer. No. 09/395,694, filed Sep. 13, 1999, now U.S. Pat. No. 6,226,392,which is a continuation of U.S. patent application Ser. No. 08/805,856,filed Mar. 3, 1997, which is now abandoned, which is acontinuation-in-part of U.S. patent application Ser. No. 08/701,974filed on Aug. 24, 1996, now U.S. Pat. No. 6,031,930.

FIELD OF THE INVENTION

[0002] This invention relates to a method of, and an apparatus for,acquiring and constructing tiled digital images from a specimen on asupport, such as a microscope slide, and for storing, and transferringthe image for viewing by another at a local or remote location.

BACKGROUND OF THE INVENTION

[0003] In the image analysis and quantification of DNA from tissuesections as disclosed in U.S. Pat. No. 4,741,031, and also especially inthe immunohisto-chemistry assays on the kinds of cell analysis systemsdisclosed in U.S. Pat. Nos. 5,086,476; 5,202,931; and 5,252,487 issuedto Bacus, there is a problem of first locating the cancer regions foranalysis under low power and then remembering them when performing theanalysis under higher power. Specifically, the problem is that once themicroscope is set up for quantification by image analysis under, e.g.40×, where all of the diaphragms are set and light adjusted, etc., ifthe operator needs to move to another tissue area, it is first desirableto locate it at e.g. 10×. In fact, often regions can only be located atthis power. In order to do so however, all of the settings (diaphragms,light levels, wavelengths of light, etc.) have to be changed to view thetissue at this magnification. Currently, there is no way to ensure thatone could go back to the settings at the previous 40× magnification andcontinue on with the quantitative image analysis of that same specimen.This necessitates finding those areas under 40×, without changingobjectives, which is very slow and time-consuming, and often importantcancer areas can be missed.

[0004] Also, another problem with tissue analysis, at its currentstate-of-the-art, is that it is not completely automated, for example,with regard to finding structural regions such as glands, basal layersor other important diagnostic regions. However, as set forth in myco-pending patent application Serial No. 701,974, filed Aug. 23, 1996,if these regions are located, important very sensitive diagnosticmeasurements can be performed, which patent application is incorporatedby reference as if fully reproduced herein. For example, as disclosed inthe aforesaid patent application, assays are made of a variety of tissuetypes, both human and animal for analysis of neoplasia in tissue, forpre-invasive cancer in tissue, and the effects on the tissue ofchemopreventive agents. A quantitative analysis by image processingtechniques is performed on tissue types, having various architecturalfeatures, such as breast tissue, colon tissue, prostate tissue,esophageal tissue, skin tissue, cervix tissue, etc. These tissues havedifferent morphologies, and they undergo different neoplasias usuallyresulting from a cellular mutation, as maybe enhanced by a carcinogen,or resulting from a cellular proliferation rate enhanced by hormones,growth factors, or other inducers of abnormal tissue growth. Often it isdesired to quantify small changes in the neoplasia when it is incipientor through a series of analyses performed at close time intervals tomeasure whether the neoplasia progression is increasing or has beenslowed, stopped or regressed.

[0005] Usually, the tissue specimens are cut to expose the basal layerfor review under the microscope. Typically, the quantitativemeasurements are performed at 40× to obtain 100 to 400 tissue images.The 40× objective provides a narrow field of view of a very smallportion of the entire basal layer. Often, the basal layer is somewhatelongated and generally linear such as a basal layer in a rat esophagus;and the analysis of the basal layer requires examining it along itslength. The basal layer in a mouse colon is more in the form of anirregular, circular shape; and the analysis of this basal layer requirestraveling about this circular shape. In breast tissue samples, suspectedtumor areas may be at widely-spaced points in the stained tissue; andone wants to be able to navigate and travel to these specific suspectedareas and to do the 40× analysis at these areas in an efficient manner.There is a need to allow an experienced operator to interact with theanalysis to locate and identify such regions in an interactive manner.Especially, an interactive manner that would be familiar and consistentwith the use of a microscope manually, with high power magnification andlow power magnification simultaneously available, but performed on acomputer terminal. Such a level of interaction is different than theinteraction with the system disclosed in the above-listed Bacus patents.

[0006] There is a need to take the level of interaction to a higherlevel and let each component, the human and the computer, perform thepart that it does best, in the most cost-effective manner. There areavailable on the market computer-controlled, automated microscopes suchas those sold by Carl Zeiss, Inc., Thornwood, N.J., under the nameAxioplan 2 for taking photographic images of a specimen in themicroscopic field of view. Those particular microscopes havecomputer-controlled and automatically adjusted subsystems, such as anillumination subsystem, a focusing subsystem, a diaphragm or opticalstops subsystem, an objective lens subsystem, or a filtering subsystem.As an operator selects changes from one objective lens, such as oneproviding low magnification, e.g., 4×, to a higher magnification, e.g.,40×, the computer-automated system will turn the lens turret to switchin the high magnification automatically and adjusts the lens and alsoautomatically adjusts the illumination to eliminate glare and to providethe proper light illumination including light density. Further, thefocus is adjusted, and the proper diaphragm openings are automaticallyreset. Thus, the computer-controlled, automated subsystems automaticallyreset to values stored and predetermined for each selected objectivelens and the analysis being done.

[0007] Those particular microscopes can be used to view various objectsor specimens, but are most typically used to view and to take stillphotographs of biological specimens, such as tissues and cells. Thoseparticular microscopes lack a computer-controlled X and Y stage fortranslating a specimen-carrying slide with respect to the field of viewof the selected objective lens. Currently, pathologists and others whouse such microscopes want to view the specimen images in full color orin enhanced colors using fluorescent illumination and/or monochromaticimages using the automated filter subsystems on the microscopes.Currently trained pathologists or clinicians are accustomed to manuallyadjust and have a microscope available to them to view larger areas ofthe specimen at low magnification, and then to momentarily switch in anew higher magnification lens to obtain a more highly magnified image ofa portion of the specimen viewed at low magnification. Pathologists andthose working in this area have created in themselves a desire to viewsuspect tissue through a microscope and appear to resist analysissystems that do not provide them this ability.

[0008] The microscopic field of view reduces very substantially as themagnification increases. The skill level of the clinician and/orpathologist is important to locate viewing the most suspicious areas orpoints of interest on the specimen. Sometimes, a technician will do afirst assay and analysis. A pathologist will return to the selectedpoints of interest or other points of interest for review and analysis.One concern with respect to a quantitative analysis of breast cancertissue or prostate biopsy tissue samples to pap smears or other testsfor various cancers or the like is that a particularly suspicious pointin the tissue may be overlooked and missed during the visual assay orfor selection for an automated review analysis. When observing at highmagnifications, the field of view is limited to very small area of thespecimen. Hence, the observer has difficulty in knowing and rememberingthe actual, exact location of this small periscopic view within the verylarge whole specimen.

[0009] Often, also the problem is finding or locating the tissue orcells for view at high magnification so that artifacts and/or blankspaces on the slide are not viewed. A number of approaches have beenproposed to prescreen and locate by an X and Y address the cells orsmall points of interest from a very large number of potential points ofinterest.

[0010] There are currently available commercial services forprescreening pap smears where one can mail in slides and the servicewill do a microscopic prescan at high magnification for suspected orsuspicious areas of interest which are marked and given addresslocations, and also a video tape of the slide specimen is returned bythis service to the sender. The sender then reviews the areas ofinterest located during the prescreening and/or the video tape tocomplete the analysis.

[0011] In an attempt to locate and allow review of specified points ofinterest, U.S. Pat. No. 5,428,690 to Bacus discloses a system forprescreening of a field of cells on a specimen slide at lowmagnification before the viewer. When seeing a point of interest to beviewed at high magnification, the viewer will operate a switch or thelike to select and record the address of these selected prescreenedpoints of interest. Later, these prescreened points of interest are thenbrought into position to be analyzed at high magnification. This systemis fairly or too slow for many uses.

[0012] A very expensive system is currently in use in which apathologist located at a diagnostic center is able to make a diagnosticopinion with respect to specimens under a microscope at a remote center.The pathologist at the diagnostic center manipulates robotic controls tosend telepathy signals via a special, dedicated satellite or other largebandwidth channel to control the microscope at the remote site inapproximately real time. The pathologist then can maneuver the remotemicroscope to shift the microscope's field of view and to send, bytelepathy, a highly magnified, very small image back to the pathologist.This system requires each subscriber to have a special microscopeoperable by manipulation of the robotic controls at the diagnosticcenter and a dedicated or large bandwidth channel to convey real timevideo signals and hence results in a very high cost for the assay beingdone. To assist the pathologist in staying within the specimen at theremote site, a peripheral edge or map of the specimen is made using asecond video camera and a light box or by using computerized scanningequipment to trace the outline or peripheral edge of the specimenboundaries. A small circle of light is displayed within the map of thespecimen so that the pathologist at the diagnostic center knows thelocation of the field of view of the highly magnified image within thespecimen. In a sense, the pathologist is operating in real time in thesame way that he would use his own microscope at his diagnostic centerexcept for a slight transmission delay used to transmit the videosignals of the highly magnified image over large bandwidth channel.Although the pathologist has a small map or peripheral outline of thespecimen, the pathologist's field of view of the actual specimen is onlythe small circle of view that is coming through the microscope objectivelens. This does not help the pathologist locate suspicious areas ofinterest as in a prescreening of the entire tissue. The pathologist mayswitch to the lowest magnification to get the largest field of view of asmall section of the specimen, but he never views the entire specimen atany magnification. Also, there is no image analysis quantitative testingfrom the received images at the diagnostic center; and no quantitativeassaying is done with these images at the diagnostic center.

[0013] There is a particular interest today in using the Internet systembecause it is so readily accessible by users at a low cost and using acomputer and viewing screen connected to the computer. One problem withtrying to do any transmission of digitized, microscopic, highlymagnified images over the Internet is that the bandwidth is too narrowto accommodate the tremendous amount of stored data which needs to betransmitted. There is a need for a system which would allow apathologist or another person, to be able to perform tissue analysis orquantitative assays using a standard computer terminal from a locationremote from the automated microscope.

SUMMARY OF THE INVENTION

[0014] In accordance with the present invention, a person such as apathologist at a computer terminal may control an automated microscopeto acquire on a computer screen or window images of the specimen atdifferent magnifications as selected by the person. Further, the personmay receive on the screen a low magnification, reconstructed image ofthe entire specimen to aid the person in interactively selecting pointsof interest on the specimen, such as along a basal layer of a tissuespecimen.

[0015] More specifically, and in accordance with the present invention,the microscope's small field of view limitation of a specimen isovercome by providing to the viewer a reconstructed, digitized andmagnified image of the entire specimen (or a very large portion of thespecimen) for performing a visual analysis of the entire tissue in fullcolor to aid in the selection of points of interest to be viewed at ahigher magnification. This is achieved by acquiring a large number oflow magnification images of the specimen through a microscopic scanningsystem, e.g., 35 image tiles of the specimen at 1.25×, and thenassembling and coordinating the tiles to form an overall, low magnifiedimage of the specimen, i.e., a macro image of the specimen. Preferably,the digitized macro image is reduced in size by a software system to asmaller size, e.g., a ¼ size image that is displayed on a local screenor is sent over a low band width or a high band width channel to aremote screen. Thus, the pathologist not only does not need to haveothers do a slow laborious prescreening to locate suspicious areas foranalysis or for viewing at high magnification, he can use his ownexperiences to go directly to the most suspicious areas which he sees onthe macro image. He can, on a priority basis, do the most suspiciousarea first, followed by lower priority areas of interest.

[0016] In accordance with the present invention, there is provided a newand improved automated, computer-controlled microscope that displays thelow magnification composite image of the specimen to allow the user toview and to interactively select points of interest, each of which maybe displayed at high magnification. This is achieved by providing theuser with a marker, such as a cursor or the like, to select the definedarea of interest; and to acquire reproduced, spatially adjacent highmagnification, digitized images of the selected area of interest. Morespecifically, the specimen, when it was first scanned at lowmagnification to provide a macro view of the specimen, the addresses orlocations of the tile images and/or pixels for the composite image wereacquired. Therefore, any selected region of interest in the macro imagehas locations to which the microscopic stage may be automaticallyrepositioned under a changed, higher magnification lens to acquirehigher magnification, digitized image tiles that can be assembled into amicro image. Herein, both the macro and micro images are formed ofadjacent digitized image tiles which have been coordinated to reproducespatially the original image that was scanned.

[0017] It is the high magnification images, usually at 40×, that wereanalyzed using image processing techniques as disclosed in the aforesaidpatent application, to provide an assay or numerical histological datafor the specimen.

[0018] In accordance with the preferred embodiment of the invention, thepathologist may select a larger region for analysis at high resolutionthan can be accommodated at this magnification on his high magnificationviewing screen. He can, though, view all of the adjacent, highlymagnified, digitized image tiles on this high magnification screen byscrolling up or down or right to left to shift these digitized, adjacentimage tiles into view on the screen. Thus, even at a highermagnification of a region the pathologist is able to obtain a muchlarger view than the small field for the objective lens in use ofadjacent tissue or cells to give him a broader, overall perspective ofwhat is happening or what has happened in specific section of aspecimen. For instance, a pathologist may want to see at highmagnification and to assay at this high magnification, a suspiciousarea, the pathologist can draw a mark about the area and cause it tothen be assayed and displayed.

[0019] By having displayed on a low magnification screen or split screenof the full composite area, by having the high magnification regionbeing marked on the low image screen, and by having the region beingscrolled to view adjacent high magnification images on the highmagnification screen the pathologist has available information to guidehim in the sense of helping to navigate intelligently within thespecimen in his search for cancerous tissue or the like for furtherinspection or measurement of malignant characteristics. Often, whenrestricted to a field of view of an objective microscope, thepathologist has a difficult time, in the words of a cliche, of seeingthe forest for the trees. In the present invention, the pathologist hasa full, magnified, reduced in size specimen view with the higher imagearea marked thereon to guide him and to let him see the forest. He canalso see a region of the forest at higher magnification by scrollingadjacent tree images onto the high magnification screen.

[0020] In accordance with a further aspect of the invention, the usermay elect to change to an intermediate magnification being viewed byeither switching automatically to a new objective lens and acquiring newdigitized image tiles at the intermediate magnification or by usingsoftware to reconstruct from the existing high and low magnification,digitized images a new intermediate digitized image.

[0021] The preferred, low magnification image, which is reduced in size,can be transmitted over narrow band width channels such as a local areanetwork or over the Internet through various servers and computers.Likewise, the reconstructed, high magnification images can betransmitted over such narrow band width channels. Because the microscopeis fully computer controlled, a pathologist or other person having asplit screen computer such as a PC, can be connected to the microscopeand operate it from a remote location to obtain the macro image and tonavigate to points of interest and obtain the desired micro images. Withthe present invention, there is no need for a specialized microscope ateach remote location nor for a broad band channel to send video signalsin real time between the diagnostic center and the remote location.

BRIEF DESCRIPTION OF THE DRAWINGS

[0022]FIG. 1 is a screen view of a system embodying the presentinvention showing a low magnification image of a specimen on amicroscope slide in one window, a high magnification image of a portionof the low magnification image selected by a region marker and a controlwindow;

[0023]FIG. 2 is a view of a display screen of the apparatus embodyingthe present invention showing the control window a low magnificationwindow having a plurality of high magnification micro image regionsdelineated therein and a high magnification window including one or moreof the micro image regions;

[0024]FIG. 3 is a view similar to FIG. 2 including the control windowbut also including a low magnification region from the slide showingregions marked by a histology grade or structure through automaticanalysis of tissue and a high magnification window showing markingsrelated to the grading or histology grade yielded by the automaticanalysis of tissue in combination with a window showing a numericalscore;

[0025]FIG. 4 is a block diagram of the apparatus embodying the presentinvention;

[0026]FIG. 5 is a block diagram of a portion of the apparatus shown inFIG. 4 showing details of a mechanical arrangement of a microscope;

[0027]FIG. 6 is a flow diagram related to operation of the apparatus;

[0028]FIG. 7 is a flow diagram of details of one of the steps in FIG. 6;

[0029]FIG. 8 is a display screen showing control parameters to bemanipulated thereon;

[0030]FIG. 9 is a flow chart for a region outlying routine;

[0031]FIG. 10 is a flow chart for a scanning and analyzing routine;

[0032]FIG. 11 is a schematic showing of the limits of travel of themicroscope stage with respect to the image tiles;

[0033]FIG. 11A is a perspective view of the microscope stage and steppermotors and encoders providing a closed loop drive for the motors;

[0034]FIG. 12 is a block diagram of a networked system allowing multipleworkstations to obtain access to diagnostic image information and tomanipulate such information locally at each workstation;

[0035]FIG. 12A is a view of the system described in connection with FIG.5;

[0036]FIG. 13 is a block diagram of a remote networked system fordistributing and accessing diagnostic images and data through ahypertext transport protocol based server directly or over a packetnetwork;

[0037]FIG. 14 is a view of a low magnification, reconstructed image froma basal layer of rat esophagus;

[0038]FIG. 14A is a view of a high magnification, reconstructed imagefrom a selected point of interest from FIG. 14;

[0039]FIG. 15 is a view of a low magnification image of a mouse colonhaving a basal layer;

[0040]FIG. 15A is a view of a reconstructed macro image of a mousecolon;

[0041]FIG. 16 is a schematic view of an analysis from regions of a basallayer;

[0042]FIG. 16A is a schematic view of an analysis to provide to a Zscore; and

[0043]FIG. 17 is a schematic view showing texture analysis tests forregions.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

[0044] Referring now to the drawings, and especially to FIGS. 4 and 5,apparatus for synthesizing low magnification and high magnificationmicroscopic images is shown therein and generally identified byreference numeral 10. The system includes a computer 12 which is a dualPentium Pro personal computer in combination with a Hitachi HV-C20 videocamera 14 associated with a Zeiss Axioplan 2 microscope 16. The computersystem 12 is able to receive signals from the camera 14 which captureslight from the microscope 16 having a microscope slide 18 positioned onan LUDL encoded motorized stage 20. The encoded motorized stage 20includes a MAC 2000 stage controller for controlling the stage inresponse to the computer 12. A microscope slide 18 includes a biologicalspecimen 21 which is to be viewed by the microscope and whose image isto be digitized both at low magnification and at high magnification asselected by a user. The low magnification digitized image is thendisplayed on a 21 inch Iiyama video display monitor 22 having resolutionof 1600 by 1200 to provide display screens of the type shown in FIGS. 1through 3 including a low magnification image 24, for instance, at 1.25power, a high magnification image 26, for instance at 40 power and acontrol window or image 28. The low magnification image may haveidentified therein a region 30 which is reproduced at high magnificationin high magnification screen or window 26 so that a pathologist or otheroperator of the system can review architectural regions of interest inlow magnification image 24 and simultaneously view them in highmagnification in the high magnification screen or window 26 to determinewhether the cells forming a portion of the architectural feature need beexamined further for cancer or the like or not.

[0045] The computer 10 is constructed around a PCI system bus 40 and hasa first Pentium Pro microprocessor 42 and a second pentium promicroprocessor 44 connected thereto. The system bus 40 has connected toit a PCI bus 50 and an ISA bus 52. The PCI bus 50 has a SCSI controller60 connected thereto to send and receive information from a hard disk62. The hard disk 62 also is coupled in daisy chain SCSI fashion to ahigh capacity removal disk and to a CD Rom drive 66. The hard disks 62contains the programs for operating the system for controlling themicroscope 16 and for processing the images as well as for doing aquantitative analysis of the selected portions of the histologicalspecimens being viewed on the slide 18. The system bus 40 also hasconnected to it a random access memory 70 within which portions of theprogram being executed are stored as well as a read only memory 72 forholding a bootstrap loader as well as portions of the basic input/outputoperating system. A floppy disk controller 74 is coupled to the systembus 40 and has connected to it a floppy disk drive 76 for reading andwriting information to a floppy disk as appropriate. A mouse controller80 is coupled to the system bus and has a mouse 82 which operates as apointing device for controlling manipulations on the screen 22 andwithin the windows 24, 26 and 28. A keyboard controller 90 is connectedto the system bus and has a keyboard 92 connected thereto. The keyboard92 may be used to send and receive alpha numeric signals to otherportions of the computer. An audio controller 100 has a plurality ofspeakers 102 and a microphone 104 connected thereto for audio input andoutput and is coupled to the system bus 40. A network interface, such asa network interface card 104, is connected to the system bus and canprovide signals via a channel 106 to other portions of a network orinternet to which the system may be connected. Likewise, signals can besent out of the system through a modem 110 connected to the ISA bus 52and may be sent via a channel 112, for instance, to the internet. Aprinter 116 is connected via a parallel I/O controller 118 to the systembus in order to provide printouts as appropriate of screens and otherinformation as it is generated. A serial I/O controller 122 is connectedto the system bus and has connected to it a camera controller 124 whichis coupled to CCD sensors 126 in the cameras. The CCD sensors 126 supplypixel or image signals representative of what is found on the slide 18to an Epix pixci image acquisition controller 130 coupled to the PCI bus50.

[0046] The microscope 16 includes a base 140 having a stage 20positioned thereon as well as an objective turret 142 having a pluralityof objectives 144, 146 and 148 thereon. The objective 144, for instance,may be of 1.25× objective. The objective 146 may be a 20× objective. Theobjective 148 may be a 40× objective. Signals from the camera sensorsand controller are supplied over a bus 128 to the image acquisitionsystem where they are digitized and supplied to the PCI bus for storagein RAM or for backing storage on the hard disk 62.

[0047] When a specimen is on the slide 18 the stage 20 may bemanipulated under the control of the computer through a stage controller160 coupled to the serial I/O controller 122. Likewise, a microscopecontroller 162 controls aspects of the microscope such as theillumination, the color temperature or spectral output of a lamp 168 andthe like. For instance, in normal operation, when a specimen is placedon the slide, specimen slide 18 is placed on the stage 20 in a step 200,as shown in FIG. 6, the processors 42 or 44 send a command through thesystem bus to cause the serial I/O controller 122 to signal themicroscope controller to change magnification to 1.25× in a step 202.This is done by rotating the objective turret of the Axioplan 2microscope to select the objective 144. Likewise, the controller setsthe color temperature of the lamp 168, sets a pair of neutral densityfilter wheels 170 and 172 and sets a field diaphragm 174 for the correctillumination. A condenser diaphragm 176 is also controlled and a colorfilter wheel 180 may also be controlled to apply the appropriate filtercolor to the CCD censors 126 in the camera. The entire slide is thenscanned in a step 204. The images are tiled and melded together into theoverall image 24 supplied on the screen 22 to provide the operator inthe step 206 with a visually inspectable macro image of relevant regionsof the slide of interest.

[0048] In order to provide the magnified image, the mouse may be movedto identify a marker segment or region which, for instance, may be arectangular region (as shown as 30 in FIG. 1) which will cause themicroscope to change magnification as at step 208 to 4×, 20×, 40×, etc.,by rotating the turret to bring the appropriate objective lens systeminto viewing position.

[0049] Next the user, in a step 209 a, uses the mouse to select theregion on the macro image in order to select the micro image to beviewed on the screen 22. In a step 209 b a test is made to determinewhether the user has commanded continued inspection. If the user has, atest is made in a step 209 c to determine if the magnification is to bechanged by changing the selected objective. In the event themagnification is to be changed control is transferred to the step 208.If the magnification is to remain unchanged control is transferred tothe step 209 a. In the event inspection is not to continue the regionselected is outlined for higher magnification scan in a step 209 d. In astep 209 e, a command may be received to scan or acquire the highermagnification image for display in screen 26. The image may then bearchived for later analysis, displayed or analyzed immediately.

[0050] In order to perform the magnification called for in step 208, theoverall illumination and control of the microscope will be controlled sothat in a step 210 the objective turret 142 will be rotated to place thehigher power objective above the slide 18. In a step 212 voltage to thelamp will be changed to adjust the lamp 168 to provide the properillumination and color temperature as predetermined for the selectedobjective. In a step 214, the condenser diaphragm 176 will have itsopening selected as appropriate to provide the proper illumination forthat objective. In a step 216, the filter turret 180 will select theproper light wavelength filter to be supplied to the camera sensors. Forinstance, a red, blue or green filter, as appropriate, particularly ifthe specimen has been stained. In a step 218 the field diaphragm 174will have its opening changed. In a step 220 the neutral density filterwheel 170 will select a neutral density filter and in a step 222 theneutral density filter wheel 172 will also select a neutral densityfilter. In a step 224 the X, Y and Z offsets will be used forreconstruction of the recorded image at the magnification and in a step226 the current position will be read from encoders in the stage whichare accurate to 0.10 micron.

[0051] In order to identify the selected region the mouse is moved tothat area of the region in a pointing operation in a step 240 as shownin FIG. 9. The mouse may be moved to draw a box around the regionselected. In a step 242 the X and Y screen points are computed for theedges of the regions selected and the computed image or pixel points aretranslated to stage coordinate points in order to control the stage ofthe microscope. In a step 244 a list of all of the X fields forpositioning the stage for the objective is stored in random accessmemory and may be backed up on the hard disk. The information from the Xoffsets for the objective and the stage offsets is used as well as thesize of the field to position the slide properly under the objective tocapture the micro image.

[0052] When the slide has been positioned properly, as shown in FIG. 10in a step 250 the stage is positioned for each of the X and Y coordinatevalues in stage coordinate values and the digitized image is captured bythe cameras and stored in RAM and backed up on the hard disk. The imagemay be then analyzed quantitatively in various manners such as those setforth in the previously-identified United States application. Optionallythe image may be stored for archival purposes in a step 254.

[0053] In order to override the specific control functions that takeplace as shown in FIG. 7, a screen is provided as shown in FIG. 8wherein the XY step size can be edited, the X, Y and Z offset can beedited, the lamp voltage can be selected, the neutral density filter canbe selected as well as the opening of the field diaphragm and severalother microscopic characteristics. FIG. 8 is a view of the settings ofthe microscope objective properties of the Axioplan 2,computer-controlled microscope.

[0054] The X and Y positioning is specifically carried out as shown inFIG. 1 where the slide 18 is shown with a slide boundary 270, 272, 274and 276. Stage boundary for limits of the stage travel for purposes ofthe stage the stage can be moved all the way from an upper left handcorner of travel 276 to a lower right hand corner of travel 280. At theupper left hand bounded corner of travel 278 limits which a signal thatthe end of travel has been reached and the stage is then translated ashort distance 282 in the extra action and a short distance 284 in the Ydirection to define the first tile 288 in terms of a reference point 290at its upper left hand corner. Since the size of the macro image tile288 is known, the next macro image tile 292 may be placed contiguouswith it by moving the stage appropriately and by measuring the locationof the stage from the stage in counters without the necessity ofperforming any image manipulation. The image tiles 288 and 292 may beabutted without any substantial overlap or they may be overlappedslightly, such as a one pixel with overlap, which is negligible insofaras blurring of any adjacent edges of abutted image tiles. The upper lefthand corner 300 of the tile 292 defines the rest of 292 and other tilescan be so defined. Micro image tiles can likewise be defined so thatthey are contiguous but not substantially overlapping, as wouldinterfere with the composite image. This avoids the problems encounteredwith having to perform extended computations on digital images in aframe storer or multiple frame storage in order to match or bring theimages into contiguity without blurriness at the edges of contiguousimage tiles. It may be appreciated as shown in FIG. 2 that the low powerimage 24 has a plurality of micro images defined therein which are tiledand which are shown in higher magnification as individual tiles 312,314, 316 and the like in FIG. 2. In addition, the region 310 whenmagnified as shown in the window 26 may exceed the bounds of the windowand thus the window may include scroll bars or other means for allowingthe image 310 which is larger than the window 26 to be examined fromwithin the window 26.

[0055] The stage 200 is best seen in FIG. 11A and includes the X and Ystepper motors 279 and 281 with their respective encoders, which providea closed loop system to give the 0.1 micron accuracy versus the usual 5or 6 micron accuracy of most microscope stages without a closed loopsystem. This closed loop system and this very high accuracy allow theabutting of the tile images for both high magnification and lowmagnification images without the substantial overlap and thetime-consuming and expensive software currently used to eliminate theoverlap and blurriness at the overlapping edges of adjacent image tiles.With the precisely positioned stage and by using the tiling systemdescribed in connection with FIG. 11, where the slide is preciselypositioned relative to a center point CP for the slide, and the knownposition of point 278 is always taken from the same point, the tiles maybe positioned precisely in a horizontal row and precisely in verticalrows to reconstruct the macro image and the micro image. Thisreconstruction is done without the use, as in the prior art, ofextensive software manipulation to eliminate overlapping image tiles,horizontally or vertically or the haphazard orientation of image tiles.

[0056] Furthermore, as shown in FIG. 3, the low power window 24, highpower window 26 and control window 28 can be used in conjunction withreporting of quantitative analysis data, histograms, etc. for thespecimen being viewed; and such analysis information may be provided asa visual output in a window 320. Each of the various regions 30 that apathologist may follow in marking various features in the low powerwindow 24 and the high power window 26 may be reflected in both windowsin order that an audit trail is provided for the system.

[0057] The present invention also includes the facility for allowingremote diagnostics to occur by being able to couple the system eitherover a network communication facility to an intranet, for instance viathe network interface, or via a modem or other suitable connection, toan internet so that once the image has been scanned and stored in memoryon hard disks or other storage, remote users may be able to access thelow magnification image as well as the high magnification image and movearound within both images to make determinations as to the histologicalcharacteristics of the samples via Z scores.

[0058] An additional feature of the system includes a plurality ofnetworked workstations coupled to a first computer console 12 having adisplay screen 22 connected to the microscope 14. Satellite workstations 350 and 352 are substantially identical to the work station 12including respective computers 354 and 356 coupled to displays 358 and360. The devices can be manipulated through input devices 360 and 362which may include a keyboard, mouse and the like. Also a third devicecan be connected including a work station 370, having a display 372, acomputer 374 and an input device 376. Each of the devices is connectedover respective network lines 380, 382, 384 to the computer 12 whichtransmission may be via either net or the like. Each of the differentoperators at the physically separate viewing stations can locate regionsfrom the view of entire tissue cross sections via a macro view and labelthe regions for subsequent scanning and/or quantitative analysis. Asingle operator at the instrument station 12 can locate regions to viewthe entire tissue cross section. Those regions can be labeled forsubsequent scanning and/or quantitative analysis with subsequent reviewand physically remote viewing stations, for instance, in an operatingroom or in individual pathologists' signout areas in order to reviewanalysis results while still maintaining and reviewing the entire macroview of the tissue and/or the individual stored images from which thequantitative results were obtained. The viewing stations 350, 352 and370 can comprise desk top computers, laptops, etc. There is no need fora microscope at the network stations 350, 352 and 370.

[0059] In a still further alternative embodiment, remote workstations400, 402, 404, 406 and 408 may be connected through a server 410 whichmay be supplied via a packet switched network. The server 410 and may bea hypertext transport protocol based server of the type used for theWorld Wide Web or may be a telnet type server as used previously ininternet remote operation applications. The server 410 communicates viaa communications channel 414 with a local computer 416 having a display418 associated therewith, the local computer 416 being connected to themicroscope 420. Each of the remote work stations 400, 402, 404, 406 and408 may perform the same operations as the stations 350, 352 and 370although they do it from nearby buildings or even from around the world,thus providing additional flexibility for others to make use of thespecimen obtained and being viewed under the microscope 420. Inaddition, stored images may be disseminated through the server 410 tothe remote servers 400 through 408 for further analysis and review.

[0060] In FIG. 14, there is illustrated on screen 28 a basal layer 431 aof a cut cross-section of a rat esophagus. The basal layer is elongatedand linear in a downward direction, and the selected point of interestis shown as a box 30 on the basal layer on the composite, lowmagnification image. The high magnification image 26 of this selectedpoint of interest is shown on screen 26 in FIG. 14A. In FIG. 15 is showna mouse colon as a reconstructed, low magnification macro image 28 whichhas been reduced {fraction (1/16)}th in size. The micro image 26 isshown in FIG. 15A, and the marking therefore is shown in FIG. 15.

[0061] The analysis for texture and for morphological features used toanalyze a series of regions 30 on the elongated basal layer that wereanalyzed at high magnification are shown in FIGS. 16, 16A and 17. Themanner of doing these tests and of obtaining a Z score or grade isdisclosed in the aforesaid patent application.

[0062] While there has been illustrated and described a particularembodiment of the present invention, it will be appreciated thatnumerous changes and modifications will occur to those skilled in theart, and it is intended in the appended claims to cover all thosechanges and modifications which followed in the true spirit and scope ofthe present invention.

What is claimed is:
 1. A method of using a computer-controlled microscope imaging system for acquiring and analyzing areas of interest, said method comprising the steps of: scanning and digitizing a specimen at a low magnification through a microscope; displaying to the observer a low magnification, digitized image of the specimen; selecting a segment of the specimen from the low magnification digitized image for viewing at a higher magnification greater than the low magnification; automatically scanning the selected segment at the higher magnification and digitizing the image of the segment; and making available in real time to the view of the observer the respective, low magnification, digitized image and the high magnification, digitized image of the segment.
 2. A method for analyzing biological specimens by an image analysis system having a computer-controlled, automated microscope comprising the following steps: placing a biological specimen in a microscope at a first location for viewing at a remote location; connecting a computer terminal at a remote location over an Internet or intranet transmission channel; using a computer terminal at the remote location to control the image analysis system and microscope at the first location to acquire low magnification, adjacent image tiles of the specimen and to provide a low magnification, composite, macro image from the plurality of image tiles with the series of images being in sensing registration at their respective edges; displaying at the remote location the macro image on a screen of the computer terminal from a transmission over the transmission channel; interactively selecting at the remote location at least one point of interest on the displayed macro, specimen image for viewing at a higher magnification; sending signals from the computer terminal to control and to operate the image analysis system and the computer-controlled microscope at the first location to change objective lenses to acquire a plurality of higher magnification, adjacent image tiles and to provide a high magnification, composite micro image therefrom with the series of images being in sensing registration at their respective edges; and sending signals representative of the higher magnification, image tiles from the first location to the remote location and displaying at the remote location the micro image of the point of interest on the specimen.
 3. A method in accordance with claim 2 further comprising analyzing a layer of tissue cells in the specimen for pre-invasive neoplasia.
 4. A method in accordance with claim 3 further comprising measuring morphometric features of the specimen and measuring texture features of the specimen from the micro images.
 5. A method in accordance with claim 4 wherein interactively selecting at least one point of interest further comprises selecting a series of points along a basal layer of tissue in the specimen for analysis at high magnification.
 6. A method in accordance with claim 2 wherein interactively selecting at least one point of interest further comprises: selecting a plurality of points of interest on the macro image; and recording the location of each selected point of interest.
 7. A method in accordance with claim 2 wherein interactively selecting at least one point of interest further comprises: marking by the remote user of a region on the macro image on the screen that includes portions of adjacent, low magnification image tiles; acquiring high magnification image tile for each of marked low magnification tiles; and assembling these high magnification image tiles into a micro image and displaying them.
 8. A method in accordance with claim 2 including: simultaneously displaying on computer windows of the computer both micro and macro images of the specimen.
 9. A method in accordance with claim 2 wherein the computer terminal is at a remote location from the computer-controlled microscope, and comprising: first connecting the computer terminal at the remote location over a transmission channel to the image analysis system and the computer-controlled microscope which is located only at the first location.
 10. A method for analyzing biological specimens by an image analysis system having a computer-controlled, automated microscope comprising the following steps: placing a biological specimen in a microscope at a first location for viewing at a remote location; connecting a remote computer terminal at a remote location over an Internet or Intranet transmission channel; using the remote computer terminal at the remote location to control the image analysis system and microscope at the first location; sending signals from the remote computer terminal over the transmission channel to control and to operate the image analysis system and microscope at the first location to acquire a plurality of higher magnification, adjacent, digitized image tiles with the series of images being in registration with each other; storing the digitized image tiles for use with a server at the first computer terminal; and transmitting by the server from the first location to the remote location and displaying at the remote location images formed from the digitized image tiles transmitted over the transmission channel.
 11. A method in accordance with claim 10 comprising: connecting the server at the first location and the remote computer channel over the Internet transmission channel and sending the stored digitized image tiles over the Internet transmission channel from the first location to the remote location and operating the microscope at the first location by signals transmitted over the Internet transmission channel to the first location.
 12. A method in accordance with claim 11 comprising: providing a consulting pathologist at the remote location for operating the microscope; and sending the digitized image tiles over the Internet transmission and operating the microscope with a live transmission of digitized images tiles within a period of minutes from start of the use of the image analysis system from the remote location.
 13. An apparatus for acquiring and transmitting over an Internet or intranet transmission channel a magnified view of a substantial portion of a biological specimen and a higher magnification view of a selected point of interest for viewing at a location remote from the microscope, the apparatus comprising: an imaging system for acquiring a magnified digitized image of a substantial portion of the biological specimen and including a microscope to acquire a higher magnification, digitized, field of view images for the selected point of interest of the specimen; a control system at the remote and first locations operable by an operator at the remote location and over the Internet or intranet transmission channel to select a point of interest from the magnified, digitized image of a substantial portion of the biological specimen and causing the image system and microscope to acquire the higher magnification, digitized image for storing in the storing system for a transmitting of the higher magnification image to the remote location; a storing system for storing data representing the magnified, digitized image as data for transmission over the Internet or intranet transmission channel and for storing data representing the higher magnification image; and a display device at the remote location for displaying the transmitted higher magnification image of the selected point of interest from the transmitted data and for displaying the magnified, digitized image of a substantial portion of the biological specimen from the transmitted data.
 14. An apparatus in accordance with claim 13 wherein the imaging system for acquiring a magnified digitized image of a substantial portion of the specimen comprises: a computer controlled microscope stage system for stepping to acquire a series of adjacent, field of view images taken through the microscope and for registration with one another to provide a composite image of adjacent fields of view larger than a single field of view from the microscope.
 15. An apparatus in accordance with claim 14 wherein: an objective lens being used in the microscope having a lower magnification and resolution than an objective lens being used for the high magnification in order to obtain the low magnification, registered adjacent field of view images comprising the composite image.
 16. An apparatus in accordance with claim 13 wherein the imaging system including a microscope to acquire a higher magnification, digitized field of view image for the selected point of interest comprises: a computer controlled microscope stage system movable intermittently in steps when acquiring a series of adjacent fields of view taken through the microscope at the higher magnification and registered with one another to provide a composite image of adjacent fields of view larger than a single field of view for the point of interest.
 17. An apparatus in accordance with claim 13 wherein the imaging system comprises an X and Y storage device for storing the locations of the images of each of the fields of view.
 18. An apparatus in accordance with claim 13 wherein the display device comprises: a device operable for switching back and forth between digitized higher magnification images at several different magnifications for the same point of interest.
 19. An apparatus in accordance with claim 13 wherein the display device includes a monitor having a first portion for displaying the low magnification composite image and a second portion for displaying simultaneously the higher magnification image.
 20. An apparatus in accordance with claim 14 wherein the display device comprises: a scrolling feature to allow scrolling into view on the display device higher magnification images of adjacent fields of view.
 21. An apparatus in accordance with claim 13 comprising: a marker on the display device movable by the operator to navigate along to composite low magnification image to select a suspect area for viewing by the operator on a subsequent display on the display device.
 22. An apparatus for acquiring and transmitting over the Internet magnified microscopic views from a remotely operated microscope, the apparatus comprising: an image analysis system including a computer controlled microscope for receiving a biological specimen thereon and for acquiring field of view images at different magnifications of the specimen; a remote computer terminal at a remote location for connection over the Internet to the image analysis system and for operating the image analysis system from the remote location to acquire a high magnification field of view image from the specimen; a storage device for storing a digitized image of a high magnification field of view as data for transmission; a server associate with the image analysis system and storage device for transmission of the data to the remote location; and a display device at the remote location for displaying a low magnification image showing the morphology of the specimen for navigation thereon by the operator when remotely controlling the microscope to select a point of interest and to acquire and transmit a high magnification field of view image of the selected point of interest.
 23. An apparatus in accordance with claim 22 comprising: objective lens in the microscope being positioned to provide multiple magnification images at magnification greater than that used for the low magnification image.
 24. A method of performing a quantitative analysis on stored, digitized views of an optically magnified specimen comprising: storing a plurality of digitized images optically magnified to allow a view of a region and to allow a quantitative measurement thereof of a region larger than a single field of view; selecting a region of interest from a composite digitized image of at least a portion of the specimen; performing a quantitative analysis on the first digitized image from the region of interest; automatically requesting a second stored digitized image from the region of interest for analysis; receiving the second stored digitized image; and performing the same quantitative analysis on the second stored digitized image thereby providing a quantitative analysis with respect to the region of interest.
 25. A method in accordance with claim 24 comprising: providing a browser having a list of the digitized images for the region of interest; and the browser automatically requests the digitized images for the successive quantitative analysis.
 26. A method in accordance with claim 24 comprising: storing the digitized images for the selected region of interest in an image cache associated with the browser; and retrieving the digitized images from the cache upon a request therefor.
 27. A method in accordance with claim 24 comprising: providing digitized images for quantitative analysis that are composite views formed of a plurality of fields of view from a microscope and performing the quantitative analysis on these composite digitized images.
 28. A method in accordance with claim 24 comprising: providing a lower magnification composite view of the specimen to a user to enable the selection of a region of interest for quantitative analysis which is done on a higher magnification and resolution image.
 29. A method in accordance with claim 28 comprising: storing the digitized images at a first location; and transmitting the stored digitized images to a plurality of other locations to allow simultaneous quantitative analysis to be performed on the same region of interest.
 30. A method in accordance with claim 28 comprising: providing an Internet transmission channel and a server for distribution of the stored displayed images over the transmission channel.
 31. A method in accordance with claim 24 comprising: providing a list of digitized images for a user; selecting by the user from the list a digitized image at a lower magnification than the magnification of the digitized image used for quantification; and selecting of a region of interest by the user from the lower magnification image.
 32. A method in accordance with claim 31 comprising: providing a slide try with a list of images to be selected from by the user.
 33. A method in accordance with claim 32 comprising: transmitting the selected digitized images over the Internet to a user.
 34. A method in accordance with claim 33 comprising: a plurality of users accessing the slide tray and a plurality of users simultaneously receiving their respectively selected digitized images of their respectively selected region of interest for the quantitative analysis.
 35. A method in accordance with claim 34 comprising: providing an optical magnifier system to provide the digitizing images at the respective magnifications at a location physically removed from the locations at which the quantitative analysis is performed.
 36. A method of quantitative analysis of a biological specimen comprising: providing scanned and stored data of optically magnified digitized images from the biological specimen at a first location; providing an analysis program at a second location physically removed from the first location for performing a quantitative analysis on an optically magnified image; retrieving data of scanned and stored images upon a request from the first location for transfer to the second location; transferring the data for the digitized images to the second location and storing the retrieved data for optically magnified, digitized images at the second location; using the analysis program to perform a quantitative analysis on successive, optical magnified images obtained from the retrieved data stored at the second location; and using a control program to automatically retrieve image data for a successive second image of the specimen after performing a first quantitative analysis on a preceding image of the specimen.
 37. A method in accordance with claim 36 comprising: displaying to an observer at the second location a comprehensive image of the specimen found from the retrieved data; and selecting by the observer of a region of interest from the comprehensive view on which to perform the quantitative analysis. 